Skip to content

Solving issue with loading and saving ckpt#150

Open
black-born wants to merge 4 commits into
OHF-Voice:mainfrom
black-born:main
Open

Solving issue with loading and saving ckpt#150
black-born wants to merge 4 commits into
OHF-Voice:mainfrom
black-born:main

Conversation

@black-born

Copy link
Copy Markdown

I tried my best to not surcharge the code but if you want any modification, feel free to ask.

@phakepraful

Copy link
Copy Markdown
/usr/local/lib/python3.12/dist-packages/torch/backends/__init__.py:46: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.)
  self.setter(val)

✗ Error: [Errno 2] No such file or directory: 'https://huggingface.co/datasets/rhasspy/piper-checkpoints/resolve/main/en/en_US/hfc_female/medium/epoch%3D2868-step%3D1575188.ckpt'
Traceback (most recent call last):
  File "/content/piper1-gpl/src/piper/train/__main__.py", line 124, in process_checkpoint
    checkpoint = torch.load(input_checkpoint, weights_only=False, map_location='cpu')
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/serialization.py", line 1484, in load
    with _open_file_like(f, "rb") as opened_file:
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/serialization.py", line 759, in _open_file_like
    return _open_file(name_or_buffer, mode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/serialization.py", line 740, in __init__
    super().__init__(open(name, mode))
                     ^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'https://huggingface.co/datasets/rhasspy/piper-checkpoints/resolve/main/en/en_US/hfc_female/medium/epoch%3D2868-step%3D1575188.ckpt'

I get the above error, and it continues to run without checkpoint. when setting -ckpt_path to url -

!python3 -m piper.train fit \
  --data.voice_name "my_colab_voice" \
  --data.csv_path "/content/drive/MyDrive/tts_data/myvoice/metadata.csv" \
  --data.audio_dir "/content/drive/MyDrive/tts_data/myvoice/wavs" \
  --model.sample_rate 22050 \
  --data.espeak_voice "en-us" \
  --data.cache_dir "/content/piper_cache" \
  --data.config_path "/content/drive/MyDrive/tts_data/myvoice/my_colab_voice.json" \
  --data.batch_size 8 \
  --ckpt_path "https://huggingface.co/datasets/rhasspy/piper-checkpoints/resolve/main/en/en_US/lessac/medium/epoch%3D2164-step%3D1355540.ckpt"

If i run with --ckpt_path "/content/drive/MyDrive/tts_data/myvoice/epoch=2888-step=1320.ckpt" by mounting a gdrive in colab and setting ckpt_path to a file in gdrive, it gives me an error about not finding chekpoint at chekpoint/processed-epoch=2888-step=1320.ckpt and stops.
I can get the exact error in this case later when i rerun the colab.

Both of them work correctly in the main branch though.

But this does help in exporting, by lettiing us set --dynamo=0 .

@black-born

Copy link
Copy Markdown
Author

I think it's because I didn't take into account the fact that you could directly use an URL. My code only work if you download the checkpoint beforehand. I will fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants